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We describe a straightforward modification of frequently invoked methods for the determination 
of the statistical significance of a y-ray signal observed in a counting process. A simple criterion 
is proposed to decide whether a set of measurements of the numbers of photons registered in the 
source and background regions is consistent with the assumption of a constant source activity. 
This method is particularly suitable for immediate evaluation of the stability of the observed 
y-ray signal. It is independent of the exposure estimates, reducing thus the impact of systematic 
inaccuracies, and properly accounts for the fluctuations in the number of detected photons. The 
usefulness of the method is demonstrated on several examples. We discuss intensity changes for 
y-ray emitters detected at very high energies by the current y-ray telescopes (e.g. 1ES 0229+200, 
1ES 1959+650 and PG 1553+113). Some of the measurements are quantified to be exceptional 
with large statistical significances. 
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1. Introduction 

The asymptotic Li-Ma technique [|I]] has been traditionally used for more than thirty years in 
y-astronomy in order to confirm positive results in searching for discrete y-ray sources. In high 
energy physics, similar likelihood-based techniques are often utilized to characterize the level of 
agreement between the data and the assumption of new phenomena [2p. Also different signal- 
to-background methods or more advanced techniques based, for example, on Bayesian reasoning 
are applied |j3]. There is a large number of results based on the on-off analysis showing that a 
source producing events immersed in background is present in a suspected domain. However, 
quite different statistical tools are commonly applied in order to examine changes in a source rate. 

In this study, the standard on-off methods [|l|] are shown to be easily modified to test whether 
changes in the source activity are observed, while preserving all their important properties. Our 
way of thinking is backed by the fact that the estimation of the source flux, which is usually based 
on complex calculations of exposure and spectral features, is not necessary when just changes in the 
source intensity are to be examined. We utilize the relationship between the source and background 
regions, which is well under control in current experiments, and deal with the source flux expressed 
in terms of the background flux. Hence, based only on the numbers of recorded events in the regions 
of interest of known relative exposures, the proposed on-off analysis is independent of most of the 
details of the detection. 

The on-off method is applied to the available observational data. The recently collected very 
high energy (VHE) y-ray data on the active galactic nuclei 1ES 1959+650 {5)] is examined. We 
analyze changes in the y-ray data from the 1ES 0229+200 source [fl 0 ar| 0 from the PG 1553+113 
blazar [|9j. (Toj. [FT]]. 

2. The on-off method 

We assume a set of K independent observations with the numbers of events n on = { n{ m } and 
H 0 it = {«o|+} detected in the on-source and off-source regions and with n = H on + H 0 ff = {n l } in 
total, where i = 1,2,..., K. These on- and off-source counts are assumed to come from the Poisson 
distributions with the parameters ju on = {jj ' m } and J} () n = {/+ { ', rr }, respectively. The relationship of 
the exposures of the corresponding regions is represented by a set of on-off parameters a = {ex' }. 
The corresponding likelihood function given a set of K > 1 independent measurements of the 
numbers of events registered in the on- and off-source regions is 

K 

on,Moff) = U P n‘Mn)Pn^HU), C 2 - 1 ) 

i'=l 

where Pk{a) is the Poisson probability to observe k > 0 events with the mean a, i.e. klP/Ja) = a k e~ a . 

2.1 Test for intensity changes 

With the aim to test a constant rate of the source in a series of observations we examine whether 
the number of events registered in the on-source region when expressed in terms of background 
counts remains or does not remain constant. This hypothesis is represented by a set of K conditions 

Mon = a'j3A4f> 1 = 1,2,...,^, (2.2) 
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where /3 is an unknown source parameter. For this, we consider the likelihood ratio Rg = — 21n(^-). 

Here, (A„ n , A 0 ff) and ^f 0 = -^o(Aon,o(/3o), A 0 ff,o) = -2o(Ao, A 0 ff,o) denote, respectively, 

the likelihood function under the alternative and under the null hypothesis. These functions are 
given by Eq. ( f2~i] ) and differ by the maximum likelihood estimates of the relevant parameters. 

Consider the case of K > 2 in which more than one pair of measurements arc to be assessed 
whether they do not contradict to the hypothesis of a stable source. In a standard way [Q], ||l, the on- 
and off-source Poisson parameters fi (m = n on and ju off = n 0 ff maximize the alternative likelihood 
function. Under the null hypothesis, the maximum likelihood estimates follow from Eq. 0 
where the constraints written in Eq. ( |T2j ) are substituted, i.e. 

Aon,0 = « ! j8oAoff,o = e«(A>K, (2-3) 

where Q/(f 3) = j^,n‘ = < n + < )ff is the total number of events collected in the i-th measurement 
and the maximum likelihood estimate of the source parameter j8 is given implicitly by the equation 

K K 

£cMA)K = XXn- ( 2 - 4 ) 

i=i i=i 


„ K . K . 

If a = a', i = 1,2 ,K, this equation reduces to an explicit expression a/Jo £ n‘ )ff = £ n' m . 

i=l /—1 

Using the maximum likelihood estimates of all relevant parameters, the likelihood ratio of the 
on-off problem is T = Rk(Po) where 


K 


Rk(P) = 2£ 


1=1 


«on ln 


.2i(A)«j,n + «off. 


+ w offl n 


a'/3 


l off 


.Qi(P) «{m + "off. 


(2.5) 


Following Wilks’ theorem [Q], the statistic T is asymptotically distributed as Xk-\ with (K — \ ) 
degrees of freedom, i.e. T ~ Xk-i as n 'oa and 7I oft tend to infinity- The p-value of the test, the prob¬ 
ability that the assumption of the constant ratio for all pairs of the on- and off-source parameters 
is rejected if it holds, is then given by p = 1 — x P^_i(r) where TV-i = i (x) is the cumula¬ 
tive distribution function of the Xk-i statistic. The significance with which the hypothesis of the 
constant source activity is rejected is given by the cumulative distribution function of a standard 
Gaussian variable, <I>n = Ox (x) , i.e. by the quantile function S = <f> N 1 (I — p). 

The above described procedure can be used when dealing with one pair of on- and off- source 
counts (K = 1). The source parameter [i is chosen in advance, from other measurements or theo¬ 
retical considerations, for example. The null hypothesis is /.(<„, = ccjS/Uff where a and /J are known 
parameters. The likelihood ratio T = R\ (j3), is then asymptotically distributed with one degree 
of freedom, i.e. T ~ as n on and n 0 n tend to infinity. The resultant test statistic may be written as 


Slm = 


n>m ln 


1 ^on 

W) ^on “1“ ^off 


+ n offl n 


O'A «off "1 2 

203) n on + n oS _ J 


(2.6) 


where Q(j! 3 ) = and 5 = sgn(n on — OCj3n 0 ff)- The statistic Slm can b e assumed asymptotically as 

drawn from the standardized Gaussian distribution, Slm ~ N(0,1). The value of the sample variable 
Slm is interpreted as the asymptotic significance expressing that a | Slm | standard deviation event 
is observed above (Slm > 0) or below (Slm < 0) the given source intensity. Then, the asymptotic 
p-value for an excess or deficit is /;i m = 1 — T’nO Slm |). Choosing /3 = 1, the hypothesis of no 
source (/i on = 0£ju. o ff) is tested [[!]]. 
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Source 

Experiment 

Period 

K 

A) 

T 

p-value 

S 

{P-,P+)l a 

1ES 1959+650 

VERITAS 

2012 

3 

4.89 

173.5 

9.35 x 10~ n 

6.37 


1ES 0229+200 

HESS 

2005-2006 

2 

1.20 

0.0 

0.960 

0 

(1.11,1.29) 


VERITAS 

2009-2010 

4 

1.53 

12.2 

6.80 x 10~ 3 

2.47 

(1.46,1.60) 



2010-2011 

5 

1.15 

7.5 

0.110 

1.23 

(1.02,1.30) 



2011-2012 

5 

1.22 

2.1 

0.716 

0 

(0.94,1.56) 



2009-2012 

14 

1.34 

52.0 

1.36 x 10~ 6 

4.69 


PG 1553+113 

MAGIC 

2005-2006 

2 

1.12 

0.2 

0.673 

0 

(1.07,1.18) 


HESS 

2005-2006 

4 

1.16 

1.5 

0.693 

0 

(1.10,1.22) 


VERITAS 

2010-2012 

3 

1.60 

181.0 

1.02 x IQ- 10 

6.36 



Table 1: Results of on-off analysis. The identification of objects, experiments and observational periods are 
listed in the first three columns. The numbers of on- and off-source pairs, maximum likelihood estimates of 
the source parameter (j3o), test statistics of the constant intensity hypothesis (T), corresponding p-values and 
significances ( S ) are summarized in the following five columns. The rightmost column contains, if possible, 
confidence intervals for the source parameter fi at a 3 cr level. 


2.2 Confidence intervals for intensity 


Suppose that the statistical test described in Section 2.1 failed to reject the constant source in¬ 
tensity at a given level of significance. Then, one may construct a confidence interval of the source 
parameter at a higher level of confidence that covers its true value with the probability correspond¬ 
ing to this confidence level. For this, we adopted the method of the profile likelihood ratio. We 


assumed that the hypothesis given in Eq. ( |2.2| ) is satisfied and kept the source parameter /3 as a 
tested parameter. The likelihood ratio is then given by the likelihood function (U on ■ A off) 

under the alternative and by the profile likelihood function given the source parameter /3, i.e. 
Jz?o = JS?o(jU on (/3),/T off (/3)) = jS5o(/3,/l ofT (j3)). In the alternative model, we used the maximum 
likelihood estimates of the on- and off-source parameters, Jl on = n on and /2 off = n 0 ff. The maxi¬ 
mum likelihood estimates of the off-source parameters given /3 are a'(5 fl' oS (l5) = Qi(P)n'. 

Putting all these estimates into the likelihood ratio, we obtained that a statistic C(/3) = Rg{l 3) 
follows asymptotically a y} distribution with K degrees of freedom, i.e. C(/3 ) ~ Xk- The con¬ 
fidence interval of the source parameter /3 at a level of significance p is constructed as a com¬ 
plement of a critical domain of the hypothesis test. This way, /3 € (/3_,/3+) at a confidence 
level CL = 1 — p when the quantile function of the Xk distribution exceeds the statistic C(/3), 
i.e.^ 1 (l-p) > C(/3). Therefore, the endpoints of the confidence intervals are given by 


C(M=C(M='Vk\'-p)- 


(2.7) 


Note that the above introduced construction of the confidence interval holds for any number of 
pairs of on- and off-source counts, i.e. K > 1. 
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MJD - 56000 


Figure 1: Confidence intervals at a 3cr level of significance for the source parameter /3 of 1ES 1959+650 are 
depicted as grey boxes. The exceptional measurement is shown as a line segment with circles. Each of these 
limits is derived from a single on-off measurement. Information about MJD of the detection is included [j6|]. 
The maximum likelihood estimate of the source parameter is visualized by a blue line. 


3. Examples 


Within the proposed on-off method we examined VHE data for three objects. Specifically, 
we calculated test statistics T as well as corresponding /j-valucs and significances S when testing 
intensity changes in a series of on-off measurements, see Section 2.1. If possible, we give confi¬ 
dence intervals for the ratio of the numbers of the on-source and background events, as described 
in Section 22 Our results are summarized in Table [I] and in Figs. [I]-[|. 


3.1 1ES 1959+650 

The blazar 1ES 1959+650 was observed by the VERITAS setup between April 17, 2012 and 
June 1, 2012 (MJD 56034 and 56079) [^]. Typically, hundreds of VHE y-ray events (E > 315GeV) 
were registered in each of the three reported observations. A steady VHE flux of photons was 
rejected based on the light curve analysis with a very small p-value [7]. 

Results of our on-off analysis are shown in Table [I] and in Fig. |T[ The on-off data taken from 
Ref. [|7i] implies that the conjecture of a constant flux is rejected at a significance level of .S' ~ 6.37 
(p-value ~ 10 10 ). For the three individual observations we also provide in Fig. [I] confidence 
intervals for the source parameter /3 at a 3a level of confidence. None of derived limits corresponds 
to the overall value of the source parameter (j8o ~ 4.89) at a 3a level of confidence. We learned 
that it suffices to judge the on-off data in order that changes of the source intensity were detected 
from the 1ES 1959+650 object. 
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Figure 2: Confidence intervals at a 3cr level of significance for the source parameter p of 1ES 0229+200 
are depicted. Lines with full points and red box represent yearly measurements done by the H.E.S.S. in¬ 
strument |i8!]. Lines with circles and red boxes refer to the monthly observations during the 2009-2012 
VERITAS campaign |j7j. Blue lines show the maximum likelihood estimates of the source parameters. 

3.2 1ES 0229+200 

We employed the modified on-off scheme to investigate changes of the VHE y-ray intensity 
associated with the BL Lac object 1ES 0229+200. The data used for this purpose comprise the 
results of the 2005-2006 observations by the H.E.S.S. instrument [j8[] as well as the 2009-2012 
campaign of the VERITAS collaboration 0. The numbers of registered on- and off-source events 
in two yearly H.E.S.S. measurements (E > 580GeV) were above two hundred in 2005 and even 
above one thousand in 2006. The three years campaign (E > 300GeV) conducted by the VERITAS 
experiment provided 14 month-long data sets with the numbers of events ranging from tens to 
several thousands. 

Our on-off results are in agreement with the conclusion of the H.E.S.S. collaboration [[8]. 
When examining these two measurements we found no support for the claim that the source inten¬ 
sity changed between 2005 and 2006, see Table [I] and Fig. |2| 

The evidence for variability present in the data sets has been advocated by the VERITAS team 
using the analysis of light curve [i7|] . Our on-off results presented in Table [j] allow us to soundly 
rule out that the source emission registered by the VERITAS setup is consistent with the long-term 
baseline intensity (j8o = 1.34) at a high level of significance, S ~ 4.69 (/;-value ~ 10~ 6 ). The 
extraordinary nature of the 2009-2010 VERITAS observations is also documented in Fig. [2] where 
the time sequences of the /3-intervals are depicted. While most of the monthly data are consistent 
with background, the first monthly period in the 2009-2010 campaign is easily linked with a 3a 
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Figure 3: Confidence intervals at a 3o level of significance for the source parameter p of PG 1553+113 are 
shown. Green and red boxes are for estimates using the MAGIC and H.E.S.S. data |P[ [T()| |. Grey boxes are 
for three individual on-off measurements reported by the VERITAS team [|TT]]. Blue lines correspond to the 
maximum likelihood estimates of the source parameters. 


excess of measured counts above the three-years baseline activity. 


3.3 PG 1553+113 

The observations of BL Lac PG 1553+113 were performed by the MAGIC, H.E.S.S. as well 
as VERITAS instruments [g, 0, Oi- The light curve measured by the MAGIC telescopes in 
2005 and 2006 (E > 200 GeV) showed no significant variations on a daily time scale [9|]. Also, 
no evidence for the flux variation was detected during the 2 years of observations of the BL Lac 
PG 1553+113 by the H.E.S.S. setup (E > 200GeV) in the 2005-2006 campaigns [|To|] . The reported 
data set collected in the years 2010, 2011 and 2012 by the VERITAS instrument [ |TI| ] contains three 
yearly measurements of thousands of detected events. These VERITAS observations lead to the 
conclusion that the averaged flux detected in 2012 was increased by a factor 1.5 and 1.9 with respect 
to the 2010 and 2011 observations, respectively. 

In our on-off analysis presented in Table [I] and in Fig. |3[ we did not find any indication of 
intensity changes in the data collected by the MAGIC and H.E.S.S. telescopes. On the other hand, 
when the on-off test is performed with the VERITAS data, a clear - signature of an extraordinary 
observation registered in 2012 is returned. This finding exhibits itself in a significance of S « 6.36 
(p-value ~ 10 l0 ) when the on-off test of the constant source activity is used. 
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4. Conclusions 

We studied changes of a signal in a chosen region when individual measurements of the num¬ 
ber of source events arc accompanied by events due to background. Based on the properties of 
the Poisson process, we introduced the asymptotic on-off test for the hypothesis that assumes a 
constant ratio of the source and background activity in the on-source region. We described the con¬ 
struction of confidence intervals for the source intensity when expressed in terms of background. 

The modified significance formulas were applied to the on-off data collected from three VHE 
y-ray emitters. We showed that, independently of the previous analysis, changes of the source 
activity arc reliably quantified using on- and off-source counts. The proposed on-off scheme is not 
only supported by convincing statistical motivation, but also relatively easy to use, yet sufficiently 
general. Its independence of the complex estimation of detection conditions makes it suitable for 
exploring different types of intensity changes in different contexts. 
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